A FAMILY OF MINIMUM RENYI’S ERROR ENTROPY ALGORITHM FOR INFORMATION PROCESSING By SEUNGJU HAN A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY

of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy A FAMILY OF MINIMUM RENYI’S ERROR ENTROPY ALGORITHM FOR INFORMATION PROCESSING By Seungju Han August 2007 Chair: Jose C. Principe Major: Electrical and Computer Engineering Adaptive systems are self-adjusting and seek the optimum in a continuous way, thus becoming less dependent on a priori knowledge. However, the input signal statistics play an important role in selecting the appropriate cost function optimization. Recently, the error entropy criterion with nonparametric estimator for Renyi’s quadratic definition has been proposed as an alternative for mean square error (MSE) in supervised adaptation by Principe, Erdogmus and coworkers. For instance, minimum error entropy (MEE) had been shown as a more robust criterion for dynamic modeling and an alternative to MSE in other supervised learning applications using nonlinear systems. The major goal of our research was to extend their work, improving the MEE algorithm and demonstrating its superior performance in many practical applications that concern adaptive signal processing. We proposed four new algorithms: minimum error entropy with self adjusting step-size (MEE-SAS), normalized minimum error entropy (NMEE), fixed-point minimum error entropy (MEE-FP) and fast minimum error entropy with fast Gauss transform (fast MEE with FGT) and improved fast Gauss transform (fast MEE with IFGT). First, MEE-SAS provides a natural “Target” that is available to automatically control the algorithm step size. We attribute the self adjusting step size property of MEE-SAS to its changing curvature as opposed to MEE which has a constant curvature. Therefore, MEE-SAS has faster speed of convergence as compared to MEE algorithm for the same